The present invention is in the field of methods, systems, and computer program products for adaptive auto-pipelining for stream processing applications.
Processing within a data processing system can include various forms, including non-stream processing and stream processing. In non-stream processing, data is received, saved and later processed. In contrast, in stream processing, data is processed, as it is continuously received.
Examples of stream processing systems include large scale sense-and-respond systems, which continuously receive external signals in the form of one or more streams from multiple sources. Examples of such systems abound, ranging from systems deployed for monitoring and controlling manufacturing processes, power distribution systems, and telecommunication networks, to environmental monitoring systems, to algorithmic trading platforms, etc.